Online Clustering of Bandits with Misspecified User Models

The contextual linear bandit is an important online learning problem where given arm features, a learning agent selects an arm at each round to maximize the cumulative rewards in the long run. A line of works, called the clustering of bandits (CB), utilize the collaborative effect over user preferences and have shown significant improvements over classic linear bandit algorithms. However, existing CB algorithms require well-specified linear user models and can fail when this critical assumption does not hold. Whether robust CB algorithms can be designed for more practical scenarios with misspecified user models remains an open problem. In this paper, we are the first to present the important problem of clustering of bandits with misspecified user models (CBMUM), where the expected rewards in user models can be perturbed away from perfect linear models. We devise two robust CB algorithms, RCLUMB and RSCLUMB (representing the learned clustering structure with dynamic graph and sets, respectively), that can accommodate the inaccurate user preference estimations and erroneous clustering caused by model misspecifications. We prove regret upper bounds of $O(\epsilon_*T\sqrt{md\log T} + d\sqrt{mT}\log T)$ for our algorithms under milder assumptions than previous CB works (notably, we move past a restrictive technical assumption on the distribution of the arms), which match the lower bound asymptotically in $T$ up to logarithmic factors, and also match the state-of-the-art results in several degenerate cases. The techniques in proving the regret caused by misclustering users are quite general and may be of independent interest. Experiments on both synthetic and real-world data show our outperformance over previous algorithms

Online Corrupted User Detection and Regret Minimization

In real-world online web systems, multiple users usually arrive sequentially into the system. For applications like click fraud and fake reviews, some users can maliciously perform corrupted (disrupted) behaviors to trick the system. Therefore, it is crucial to design efficient online learning algorithms to robustly learn from potentially corrupted user behaviors and accurately identify the corrupted users in an online manner. Existing works propose bandit algorithms robust to adversarial corruption. However, these algorithms are designed for a single user, and cannot leverage the implicit social relations among multiple users for more efficient learning. Moreover, none of them consider how to detect corrupted users online in the multiple-user scenario. In this paper, we present an important online learning problem named LOCUD to learn and utilize unknown user relations from disrupted behaviors to speed up learning, and identify the corrupted users in an online setting. To robustly learn and utilize the unknown relations among potentially corrupted users, we propose a novel bandit algorithm RCLUB-WCU. To detect the corrupted users, we devise a novel online detection algorithm OCCUD based on RCLUB-WCU's inferred user relations. We prove a regret upper bound for RCLUB-WCU, which asymptotically matches the lower bound with respect to $T$ up to logarithmic factors, and matches the state-of-the-art results in degenerate cases. We also give a theoretical guarantee for the detection accuracy of OCCUD. With extensive experiments, our methods achieve superior performance over previous bandit algorithms and high corrupted user detection accuracy

Potential molecular mechanisms of overgrazing-induced dwarfism in sheepgrass (Leymus chinensis) analyzed using proteomic data

Abstract Background This study was designed to reveal potential molecular mechanisms of long-term overgrazing-induced dwarfism in sheepgrass (Leymus chinensis). Methods An electrospray ionisation mass spectrometry system was used to generate proteomic data of dwarf sheepgrass from a long-term overgrazed rangeland and normal sheepgrass from a long-term enclosed rangeland. Differentially expressed proteins (DEPs) between dwarf and normal sheepgrass were identified, after which their potential functions and interactions with each other were predicted. The expression of key DEPs was confirmed by high-performance liquid chromatography mass spectrometry (HPLC–MS) using a multiple reaction monitoring method. Results Compared with normal sheepgrass, a total of 51 upregulated and 53 downregulated proteins were identified in dwarf sheepgrass. The amino acids biosynthesis pathway was differentially enriched between the two conditions presenting DEPs, such as SAT5_ARATH and DAPA_MAIZE. The protein–protein interaction (PPI) network revealed a possible interaction between RPOB2_LEPTE, A0A023H9M8_9STRA, ATPB_DIOEL, RBL_AMOTI and DNAK_GRATL. Four modules were also extracted from the PPI network. The HPLC–MS analysis confirmed the upregulation and downregulation of ATPB_DIOEL and DNAK_GRATL, respectively in dwarf samples compared with in the controls. Conclusions The upregulated ATPB_DIOEL and downregulated DNAK_GRATL as well as proteins that interact with them, such as RPOB2_LEPTE, A0A023H9M8_9STRA and RBL_AMOTI, may be associated with the long-term overgrazing-induced dwarfism in sheepgrass

Additional file 1: of Potential molecular mechanisms of overgrazing-induced dwarfism in sheepgrass (Leymus chinensis) analyzed using proteomic data

The identified 1022 proteins that had high credibility (FDR < 0.01); 104 differentially expressed proteins between long-term overgrazed rangeland (GZ) and adjacent long-term enclosed rangeland (NG) groups; all results of Gene Ontology (GO) and Kyoto Encyclopedia of Genes and Genomes (KEGG) pathway enrichment analyses. (XLS 942 kb